[tests] Adjust UrlSessionTests to ignore more failure scenarios in CI. by rolfbjarne · Pull Request #17649 · dotnet/macios

rolfbjarne · 2023-03-01T07:52:45Z

Should fix random test failures like this:

MonoTouchFixtures.Foundation.UrlSessionTest
    [FAIL] CreateDataTaskAsync :   CreateDataTask a Exception
        Expected: null
        But was:  <Foundation.NSErrorException: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={_kCFStreamErrorCodeKey=57, NSUnderlyingError=0x6000011f4990 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={NSErrorPeerAddressKey=<CFData 0x600003deaa80 [0x7ff865b1f1c0]>{length = 16, capacity = 16, bytes = 0x100201bb6007a9b70000000000000000}, _kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <5E766082-B379-491A-BCB0-EA1B36E54A8B>.<15>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <5E766082-B379-491A-BCB0-EA1B36E54A8B>.<15>"), NSLocalizedDescription=The network connection was lost., NSErrorFailingURLStringKey=https://www.microsoft.com/, NSErrorFailingURLKey=https://www.microsoft.com/, _kCFStreamErrorDomainKey=1}
            at MonoTouchFixtures.Foundation.UrlSessionTest+<>c__DisplayClass1_0.<CreateDataTaskAsync>b__0 () [0x00039] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/monotouch-test/Foundation/UrlSessionTest.cs:64 >
            at MonoTouchFixtures.Foundation.UrlSessionTest.CreateDataTaskAsync () [0x000b7] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/monotouch-test/Foundation/UrlSessionTest.cs:71

Should fix random test failures like this: MonoTouchFixtures.Foundation.UrlSessionTest [FAIL] CreateDataTaskAsync : CreateDataTask a Exception Expected: null But was: <Foundation.NSErrorException: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={_kCFStreamErrorCodeKey=57, NSUnderlyingError=0x6000011f4990 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={NSErrorPeerAddressKey=<CFData 0x600003deaa80 [0x7ff865b1f1c0]>{length = 16, capacity = 16, bytes = 0x100201bb6007a9b70000000000000000}, _kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <5E766082-B379-491A-BCB0-EA1B36E54A8B>.<15>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <5E766082-B379-491A-BCB0-EA1B36E54A8B>.<15>"), NSLocalizedDescription=The network connection was lost., NSErrorFailingURLStringKey=https://www.microsoft.com/, NSErrorFailingURLKey=https://www.microsoft.com/, _kCFStreamErrorDomainKey=1} at MonoTouchFixtures.Foundation.UrlSessionTest+<>c__DisplayClass1_0.<CreateDataTaskAsync>b__0 () [0x00039] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/monotouch-test/Foundation/UrlSessionTest.cs:64 > at MonoTouchFixtures.Foundation.UrlSessionTest.CreateDataTaskAsync () [0x000b7] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/monotouch-test/Foundation/UrlSessionTest.cs:71

dalexsoto

neat!

vs-mobiletools-engineering-service2 · 2023-03-01T15:12:30Z

🔥 Failed to compare API and create generator diff 🔥

Error: 'make' failed for the hash ac1dad4.

Pipeline on Agent
Hash: fae8ceb98659083faf4944cb696965041650a42f [PR build]

vs-mobiletools-engineering-service2 · 2023-03-01T16:01:17Z

❌ [PR Build] Tests on macOS M1 - Mac Ventura (13.0) failed ❌

Failed tests are:

xammac_tests

Pipeline on Agent
Hash: fae8ceb98659083faf4944cb696965041650a42f [PR build]

vs-mobiletools-engineering-service2 · 2023-03-01T16:02:29Z

💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻

✅ All tests on macOS M1 - Mac Big Sur (11.5) passed.

Pipeline on Agent
Hash: fae8ceb98659083faf4944cb696965041650a42f [PR build]

vs-mobiletools-engineering-service2 · 2023-03-01T16:08:52Z

📚 [PR Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent XAMBOT-1107.Monterey'
Hash: fae8ceb98659083faf4944cb696965041650a42f [PR build]

mandel-macaque

Small nit.

mandel-macaque · 2023-03-01T16:25:02Z

+	public static void IgnoreInCIIfNetworkConnectionLost (Exception ex)
+	{
+		// <Foundation.NSErrorException: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo ...
+		if (!(ex is NSErrorException nex))


Why not using 'is not'. Works the same and will set nex for the follow up code.

Suggested change

if (!(ex is NSErrorException nex))

if (ex is not NSErrorException nex)

vs-mobiletools-engineering-service2 · 2023-03-02T09:28:41Z

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: simulator tests.

🎉 All 225 tests passed 🎉

Tests counts

✅ bcl: All 69 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests: All 1 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 7 tests passed. Html Report (VSDrops) Download
✅ framework: All 8 tests passed. Html Report (VSDrops) Download
✅ generator: All 2 tests passed. Html Report (VSDrops) Download
✅ interdependent_binding_projects: All 7 tests passed. Html Report (VSDrops) Download
✅ install_source: All 1 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ introspection: All 8 tests passed. Html Report (VSDrops) Download
✅ linker: All 65 tests passed. Html Report (VSDrops) Download
✅ mac_binding_project: All 1 tests passed. Html Report (VSDrops) Download
✅ mmp: All 2 tests passed. Html Report (VSDrops) Download
✅ mononative: All 12 tests passed. Html Report (VSDrops) Download
✅ monotouch: All 25 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ mtouch: All 1 tests passed. Html Report (VSDrops) Download
✅ xammac: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 8 tests passed. Html Report (VSDrops) Download
✅ xtro: All 2 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: fae8ceb98659083faf4944cb696965041650a42f [PR build]

rolfbjarne · 2023-03-02T11:21:40Z

/sudo backport release/7.0.2xx

rolfbjarne · 2023-03-02T11:21:50Z

/sudo backport release/7.0.1xx-xcode14.2

rolfbjarne · 2023-03-02T11:21:56Z

/sudo backport xcode14.2

rolfbjarne · 2023-03-02T11:22:04Z

/sudo backport release/6.0.4xx-xcode14.2

vs-mobiletools-engineering-service2 · 2023-03-02T11:22:12Z

Backport Job to branch release/7.0.2xx Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2023-03-02T11:22:13Z

Backport Job to branch release/7.0.1xx-xcode14.2 Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2023-03-02T11:22:20Z

Backport Job to branch xcode14.2 Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2023-03-02T11:22:32Z

Backport Job to branch release/6.0.4xx-xcode14.2 Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2023-03-02T11:24:06Z

Oh no! Backport failed! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=7409823 for more details.

vs-mobiletools-engineering-service2 · 2023-03-02T11:24:10Z

Oh no! Backport failed! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=7409824 for more details.

vs-mobiletools-engineering-service2 · 2023-03-02T11:24:18Z

Oh no! Backport failed! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=7409825 for more details.

vs-mobiletools-engineering-service2 · 2023-03-02T11:24:26Z

Oh no! Backport failed! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=7409826 for more details.

dotnet#17649) Should fix random test failures like this: MonoTouchFixtures.Foundation.UrlSessionTest [FAIL] CreateDataTaskAsync : CreateDataTask a Exception Expected: null But was: <Foundation.NSErrorException: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={_kCFStreamErrorCodeKey=57, NSUnderlyingError=0x6000011f4990 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={NSErrorPeerAddressKey=<CFData 0x600003deaa80 [0x7ff865b1f1c0]>{length = 16, capacity = 16, bytes = 0x100201bb6007a9b70000000000000000}, _kCFStreamErrorCodeKey=57, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <5E766082-B379-491A-BCB0-EA1B36E54A8B>.<15>, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalDataTask <5E766082-B379-491A-BCB0-EA1B36E54A8B>.<15>"), NSLocalizedDescription=The network connection was lost., NSErrorFailingURLStringKey=https://www.microsoft.com/, NSErrorFailingURLKey=https://www.microsoft.com/, _kCFStreamErrorDomainKey=1} at MonoTouchFixtures.Foundation.UrlSessionTest+<>c__DisplayClass1_0.<CreateDataTaskAsync>b__0 () [0x00039] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/monotouch-test/Foundation/UrlSessionTest.cs:64 > at MonoTouchFixtures.Foundation.UrlSessionTest.CreateDataTaskAsync () [0x000b7] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/monotouch-test/Foundation/UrlSessionTest.cs:71

…17677) Also add some exception handling. Hopefully fixes more issues with broken network in CI. Backport of #16945 and #17649. Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

… tests. (#17675) Also add some exception handling. Hopefully fixes more issues with broken network in CI. Backport of #16945 and #17649. --------- Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

Also add some exception handling. Hopefully fixes more issues with broken network in CI. Backport of #16945 and #17649. Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

… tests. (#17674) Also add some exception handling. Hopefully fixes more issues with broken network in CI. Backport of #16945 and #17649. --------- Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

This comment has been minimized.

Sign in to view

dalexsoto approved these changes Mar 1, 2023

View reviewed changes

Handle another case.

fae8ceb

rolfbjarne mentioned this pull request Mar 1, 2023

[net8.0] Merge main into net8.0. #17637

Merged

mandel-macaque approved these changes Mar 1, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

This was referenced Mar 2, 2023

[release/7.0.1xx-xcode14.2] [Sim] Bump min simulator versions for ventura. #17655

Merged

[release/6.0.4xx-xcode14.2] [Sim] Bump min simulator versions for ventura. #17653

Merged

haritha-mohan approved these changes Mar 2, 2023

View reviewed changes

rolfbjarne mentioned this pull request Mar 2, 2023

[ObjCRuntime] Make BlockLiteral a ref struct in XAMCORE_5_0. #17658

Merged

rolfbjarne merged commit d75b697 into dotnet:main Mar 2, 2023

rolfbjarne deleted the ci-intermittent-network-failures branch March 2, 2023 09:38

rolfbjarne mentioned this pull request Mar 2, 2023

[generator] The trampoline name is always 'Invoke', so simplify code accordingly. #17657

Merged

rolfbjarne mentioned this pull request Mar 2, 2023

[release/6.0.4xx-xcode14.2] [tests] Allow for timeouts in CI in a few tests. #17674

Merged

rolfbjarne mentioned this pull request Mar 2, 2023

[release/7.0.1xx-xcode14.2] [tests] Allow for timeouts in CI in a few tests. #17675

Merged

rolfbjarne mentioned this pull request Mar 2, 2023

[xcode14.2] [tests] Allow for timeouts in CI in a few tests. #17676

Merged

rolfbjarne mentioned this pull request Mar 2, 2023

[release/7.0.2xx] [tests] Allow for timeouts in CI in a few tests. #17677

Merged

rolfbjarne mentioned this pull request Mar 7, 2023

[tests] Fix NSTextInputClient tests on macOS Ventura. #17713

Merged

	if (!(ex is NSErrorException nex))
	if (ex is not NSErrorException nex)

Uh oh!

Conversation

rolfbjarne commented Mar 1, 2023

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

dalexsoto left a comment

Choose a reason for hiding this comment

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 1, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 1, 2023

❌ [PR Build] Tests on macOS M1 - Mac Ventura (13.0) failed ❌

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 1, 2023

💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 1, 2023

📚 [PR Build] Artifacts 📚

Packages generated

Uh oh!

mandel-macaque left a comment

Choose a reason for hiding this comment

Uh oh!

mandel-macaque Mar 1, 2023

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Mar 2, 2023

🚀 [CI Build] Test results 🚀

Test results

Tests counts

Uh oh!

rolfbjarne commented Mar 2, 2023

Uh oh!

rolfbjarne commented Mar 2, 2023

Uh oh!

rolfbjarne commented Mar 2, 2023

Uh oh!

rolfbjarne commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

vs-mobiletools-engineering-service2 commented Mar 2, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants